This invention relates to an image encoding apparatus, an image decoding apparatus, an image encoding method, an image decoding method, an image encoding program recording medium and an image decoding program recording medium.
The image encoding technology has a long history. There has been established excellent standard proposals such as ITU-T H.261, ITU-T H263, ISO MPEG1/2 and so on. Roughly speaking, the image encoding method has two approaches: an encoding method using the orthogonal transform and a prediction encoding method encoding the error of predicted values with the use of the prediction function. Although the encoding method using the orthogonal transform needs complicated calculation, when encoded signals of small bit numbers are obtained, it is possible to keep better picture quality than the prediction encoding method. The ordinary encoding method using orthogonal transform such as JPEG, MPEG and the like utilizes the DCT (Discrete Cosine Transform). Though it is known that DCT enables encoding by a small number of bits, it has own problems in that it needs high-precision multiplication, resulting in complicated calculation, and in that the reversible encoding is impossible. Accordingly, DCT calculation can not be used in the fields in which the reversibility is required.
As opposed to this, the prediction encoding method needs simple calculations and can do the reversible encoding. MMR (Modified Modified Read) used in facsimiles is famous as an image coding method having reversibility. MMR is used according to CCITT Rec.T6 xe2x80x9cFacsimile Coding Schemes and Coding Control Functions for Group 4 Facsimile Apparatusxe2x80x9d. In this method, the difference value in the horizontal direction between the change points of pixel values on the immediately previous already-coded scanning line and the change points of pixel value on the not-yet-coded scanning line is variable-length encoded. MMMR (Modified MMR) which is a further improved MMR is used as the evaluation model for MPEG4 (ISO/IEC/JTC/SC29/WG11 N1277, July 1996).
Incidentally, if image signals are separated into the objects and then the objects are processed as arbitrary shapes, image can be operated and synthesized, object by object, which leads to the effective signal-transmission. For applications which restrict bit number, by using such information, it is possible to selectively assign priority to important objects to transmit and record the same. However, the prior art technology has not taken into account the encoding of objects having arbitrary shapes. And the standardization of coding for image signals having arbitrary shapes has been proceeding in the ISO MPEG4. In MPEG4, the evaluation model called VM3.0 (printed in ISO/IEC/JTC1/SC29/WG11 N1277) is created, which is now a unique image encoding method that can encode image signals having arbitrary shapes.
An image signal having arbitrary shapes ordinarily consists of the shape information indicating the shape of an object and the pixel value information (color information) representing pixel values within an object. Concerning the shape information, the two-valued shape information indicating whether each pixel is significant(on the inside of the shape) or insignificant(on the outside of the shape), or the transparency information indicating the ratio (how much the object occludes the background) of respective pixels which is used in synthesizing with other images. When the transparency has only two levels, 0% and 100%, the shape information is identical to the transparency information and thereby the arbitrary-shape-having image signal is represented by the two of the two-valued shape information and the pixel value information.
FIG. 53 is a drawing for explaining these information. The transparency information is an information representing how much ratio of each pixel is used for synthesis when a fish shown in FIG. 53(a) is synthesized with the other image. In FIG. 53(b), there is shown the value of transparency information in the horizontal scanning line indicated by a dotted line in the figure. The outside of the fish is perfectly transparent. Here, the transparency 0 is defined as being perfectly transparent for simplification. Hence, on the outside of the fish the transparency information has a value of 0, while on the inside of the fish it has a value of non-0.
FIG. 53(c) shows the transparency which is made two-valued as having two of 0 and non-0. In FIG. 53(c), the pixels having the non-0 transparency require encoding of the pixel value information, while the pixels having the 0 transparency do not need the pixel value information, so that the two-valued transparency information is very important to the pixel value information encoding. On the other hand, the component of the transparency information which cannot be represented by two-valued information, as shown in FIG. 53(d), is multi-valued information which is called gray scale. The shape information represented by multi-valued information as described above can be treated by the waveform encoding similar as that for the pixel value information.
While performing the image encoding, the intra-frame encoding based on the spacial correlation or the temporal correlation is separately used, both of the two are employed. In the inter-frame encoding, the motion in the close frames is detected, and the motion compensation is carried out for the detected motion. The motion vector is generally used for the motion compensation. In the above-mentioned VM3.0, the intra-frame encoding and the inter-frame encoding are adaptively switched each other block by block, and the motion compensation similar as in MPEG1/2 is carried out, whereby the efficiency of encoding is improved.
As described above, when performing encoding to the image consisting of the shape information and the pixel value information, if the motion compensation encoding of a shape information is carried out using the motion vector of the pixel value information for the shape information to be used for the image synthesis, the efficiency of encoding is further improved than when the shape information is directly encoded. This is reported by ISO/IEC/JTC1/SC29/WG11 N1260 March 1996.
Further, when the motion detection and motion compensation are executed, it is considered that it is efficient that the shape information is separated into the two-valued shape information component and the multi-valued information component, and the multi-valued information component as well as the pixel value information are subjected to the same waveform encoding together, which has been actually practiced.
In the above-described prior art image encoding and the image decoding accompanying to this, there exist the following problems.
Though MMR encoding is a representative one of the reversible (loss-less) encoding as described above, because of the reversibility, it is impossible to largely improve the compression rate by allowing the visually less-important picture-quality degradation.
In addition, MMR is an intra-frame encoding method, and does not take into account the improvement of the compression rate by utilizing the inter-frame correlation. In MMR and MMMR which a modified version of MMR, only the difference between the change point of the current scanning line and the change point of the immediately previous scanning line is utilized, and the redundancy by the correlation as a straight line in the vertical direction is not sufficiently removed. Accordingly, the encoding efficiency is good when the change of the pixel value happens along the scanning line, but the encoding efficiency is bad when the change of the pixel value does not happen along the scanning line. MMR and MMMR also includes the horizontal encoding mode which does not utilize the correlation in the vertical direction at all in order to encode the pixels which can not be encoded as the difference of the change point of the immediately previous scanning line. This horizontal encoding mode has a room for further improving the efficiency with the use of the correlation in the vertical direction.
Further, in the prior art MMR and MMMR, the hierarchical image reproduction by decoding part of bit stream is impossible. The other methods in which the hierarchical image reproduction is possible have no good encoding efficiency and have demerit of increasing the encoding bit number. Accordingly, there exists no encoding method which enables the effective hierarchical image reproduction.
Further, when encoding the image consisting of shape information and image information by the motion compensation, the shape information is motion-compensated using the same motion vector as that for the image information in the prior art. However, similarly as that, if a sphere rotates, the figure drawn on the sphere moves, though the shape of the sphere does not change, the motion vector of the image information is not identical to that of the shape information. Therefore, in such a case no good encoding is carried out, which is a problem in the prior art encoding method.
Furthermore, while in VM3.0, there is a method which tries to improve the encoding efficiency by adaptively switching the intra-frame encoding and the inter-frame encoding block by block as described, the judgment against intra-frame/inter-frame encoding is based on the pixel value information similarly as in the adaptive switching in MPEG1/2, so that it is difficult to appropriately and efficiently encode the shape information which is largely different from the pixel value information in its nature.
In the light of the above-described respects, this invention is proposed and an object of this invention is to provide an image encoding apparatus, an image encoding method and an image encoding program recording medium, all of which can encode image signals efficiently. Also, another object of this invention is to provide an image decoding apparatus, an image decoding method and an image decoding program recording medium, all of which can appropriately decode the above-mentioned encoded signal encoded effectively.
In order to achieve the above-mentioned objects, a 1st aspect of this invention provides an image encoding apparatus which receives two-valued image signals as input signals and encodes pixels of the input signals changing the pixel values, comprising:
change pixel detection means for detecting the pixels changing the pixel values and outputting the result as the detected change pixels;
prediction means for predicting change pixels of the input signals based on the pixels changing the pixel values of already encoded and decoded pixels and outputting the result as predicted pixels;
difference value calculation means for calculating from the detected change pixels and the predicted change pixels the difference therebetween and outputting the same as difference values D;
rounding means for selecting Dxe2x80x2 which is in the range determined based on the given tolerance value and the difference value D and which has the minimum code length when it should be encoded and outputting it as modified difference value;
decoding means for decoding the two-valued image signal from the modified difference values Dxe2x80x2 and the predicted change pixels; and
encoding means for encoding the modified difference values Dxe2x80x2,
whereby the image encoding apparatus selects modified difference values which cause the code length of the error(difference value) to become the minimum in the prediction error equal to or smaller than the tolerance value and outputs this, resulting in reducing the bit number which is required for encoding.
A 2nd aspect of this invention provides an image encoding apparatus according to which receives two-valued image signals as input signals and encodes pixels of the input signals changing the pixel values, comprising:
change pixel detection means for detecting the pixels changing the pixel values and outputting the result as the detected change pixels;
1st prediction means for predicting change pixels of the input signals based on the pixels changing the pixel values of already encoded and decoded pixels in the frames and outputting the result as 1st predicted pixels;
1st difference value calculation means for calculating from the detected change pixels and the 1st predicted change pixels the differences therebetween and outputting the same as 1st difference values D;
2nd prediction means for predicting change pixels of the input signals, along with the motion compensation, based on the pixels changing the pixel values of already encoded and decoded pixels in reference frames and outputting the result as 2nd predicted pixels;
2nd difference value calculation means for calculating the differences between the detected change pixels and the 2nd predicted change pixels and outputting the same as 2nd difference values Dxe2x80x3;
mode selection means for calculating the code lengths of the first and second difference values Dxe2x80x2 and Dxe2x80x3 when respectively encoded, selecting the value having the shorter code length by comparing the calculated results, and outputting xe2x80x9cthe firstxe2x80x9d or xe2x80x9cthe secondxe2x80x9d, depending on the selection, as an encoding mode; and
encoding means for encoding the selected first or second difference values Dxe2x80x2 or Dxe2x80x3, and the encoded mode output by the mode selection means,
whereby the image encoding apparatus can select a signal which should have the minimum code-length by comparing the prediction based on the frame and the prediction based on the motion-compensated reference frame to output an encoded signal, resulting in reduced bit number required for encoding by the utilization of the inter-frame pixel correlation.
A 3rd aspect of this invention provides an image encoding apparatus which receives two-dimensional two-valued image signals as input signals and encodes pixels of the input signals changing the pixel values, comprising:
change pixel detection means for detecting the pixels changing the pixel values and outputting the result as the detected change pixels;
1st prediction means for predicting change pixels of the input signals based on the pixels changing the pixel values of already encoded and decoded pixels by horizontally scanning the image signals and outputting the result as 1st predicted pixels;
1st difference value calculation means for calculating the differences between the detected change pixels and the 1st predicted pixels and outputting the same as 1st difference values D;
2nd prediction means for predicting change pixels of the input signals based on the pixels changing the pixel values of already encoded and decoded pixels by scanning the image signals in the vertical direction and outputting the result as 2nd predicted pixels;
2nd difference value calculation means for calculating the differences between the detected change pixels and the 2nd predicted pixels and outputting the same as 2nd difference values Dxe2x80x3;
mode selection means for calculating the code lengths of the first and second difference values Dxe2x80x2 and Dxe2x80x3 when respectively encoded, selecting the value having the shorter code length by comparing the calculated results, and outputting xe2x80x9cthe firstxe2x80x9d or xe2x80x9cthe secondxe2x80x9d depending on the selection as an encoding mode; and
encoding means for encoding the selected first or second difference values Dxe2x80x2 or Dxe2x80x3, and the encoded mode output by the mode selection means,
whereby the image encoding apparatus can select a signal which should have the minimum code-length by comparing the prediction by the horizontal scanning and the prediction by the vertical scanning to output an encoded signal, resulting in reducing the bit number required for encoding by the utilization of local changes in the horizontal correlation and the vertical correlation of the image.
A 4th aspect of this invention provides an image encoding apparatus which receives multi-valued image signals as input signals and encodes pixels of the input signals changing the pixel values, comprising:
change pixel detection means for detecting the pixels changing the pixel values to a value above the given value and outputting the result as the detected change pixels;
prediction means for predicting change pixels of the input signals based on the pixels changing the pixel values of already encoded and decoded pixels and outputting the result as predicted pixels;
difference value calculation means for calculating the difference between the detected change pixels and the predicted pixels and outputting the same as a difference value D;
encoding means for encoding the difference values D and the pixel values of the detected change pixel; and
decoding means for decoding the multi-valued image signal from the difference values D and the pixel values of the detected change pixel,
whereby the image encoding apparatus can judge the position where the change of the pixel values is equal to or larger the threshold as the change position, and enables encoding of not only two-valued image but also multi-valued image.
A 5th aspect of this invention provides an image encoding apparatus, which receives a transparency signal indicating the ratio for the image synthesis and a pixel value signal as input signals, and encodes the input signal referring to a reference image, comprising:
1st motion vector detection means for detecting motion vectors of the pixel value signal by comparing the pixel value signal of the input signal and the pixel value signal of the reference image;
1st motion compensation means for motion-compensating the pixel value signal of the reference image using the motion vectors of the pixel value signal, and outputting a compensated pixel value signal;
1st difference value calculation means for calculating from the pixel value signal of the input signal and the compensated pixel value signal the difference therebetween, and outputting the same as 1st difference values;
1st encoding means for encoding the 1st difference values;
2nd motion vector detection means for detecting motion vectors of the transparency signal by comparing the transparency signal of the input signal and the transparency signal of the reference image;
2nd motion compensation means for motion-compensating the transparency signal of the reference image using the motion vectors of the transparency signal, and outputting the same as a compensated transparency signal;
2nd difference value calculation means for calculating from the transparency signal of the input signal and the compensated transparency signal the difference therebetween, and outputting the same as 2nd difference values;
2nd encoding means for encoding the 2nd difference values; and
3rd encoding means for encoding the motion vectors of the pixel value signal and the motion vectors of the transparency signal,
whereby the image coding apparatus motion-compensates the transparency signal using the motion vectors which are detected apart from the motion vectors of the pixel value signal and thereby approximates the input transparency signal with good precision by the motion compensation signal, resulting in reduced motion compensation error and improved encoding efficiency.
A 6th aspect of this invention provides an image encoding apparatus according to aspect 5 wherein the 2nd motion vector detection means detects the motion vectors of the transparency signal by comparing the transparency signal of the input signal and the transparency signal of the reference image in the vicinity of the motion vectors detected by the 1st motion vector detection means,
whereby the motion vectors of the transparency signal are detected only in the vicinity of the motion vectors of the pixel value signal, resulting in reduced calculation times required for detecting motion vectors, compared to the case where the calculation is performed independently of the pixel value signal.
A 7th aspect of this invention provides an image encoding apparatus according to aspect 5 wherein the 1st motion vector detection means detects the motion vectors of the pixel value signal by comparing the pixel value signal of the input signal and the pixel value signal of the reference image in the vicinity of the motion vectors detected by the 2nd motion vector detection means,
whereby the motion vectors of the pixel value signal are detected only in the vicinity of the motion vectors of the corresponding signal, resulting in reduced calculation times required for detecting motion vectors, compared to the case that the calculation is independent of the transparency signal.
An 8th aspect of this invention provides an image encoding apparatus according to aspect 5 wherein the 3rd encoding means encodes the motion vectors of the pixel value signal and the differences between the motion vectors of the transparency signal and the motion vectors of the pixel value signal,
whereby the difference vector of the motion vectors having correlation is coded and thereby the occurrence frequency of the difference vectors concentrates on in the vicinity of 0 vectors, and therefore the variable-length encoding improves the encoding efficiency and the encoding can be carried out with less bit number.
A 9th aspect of this invention provides an image encoding apparatus according to aspect 5 wherein the 3rd encoding means encodes the motion vectors of the transparency signal and the difference between the motion vectors of the transparency signal and the motion vectors of the pixel value signal,
whereby the difference vector of the motion vectors having correlation is coded and thereby the occurrence frequency of the difference vectors concentrates on in the vicinity of 0 vectors, and therefore the variable-length encoding improves the encoding efficiency and the encoding can be carried out with less bit number.
A 10th aspect of this invention provides an image encoding apparatus, which receives image signals with blocked shapes consisting of shape signals indicating the shapes of objects and whether the pixel value of pixels are significant or not and pixel value signals as input signals, and encodes the input signals referring to reference images, comprising:
1st motion vector detection means for detecting motion vectors of the pixel value signal by comparing the pixel value signal of the input signal and the pixel value signal of the reference image;
1st motion compensation means for motion-compensating the pixel value signal of the reference image using the motion vectors of the pixel value signal, and outputting the same as a compensated pixel value signal;
1st difference value calculation means for calculating from the pixel value signal of the input signal and the compensated pixel value signal the difference therebetween, and outputting the same as 1st difference values;
1st encoding means for encoding the 1st difference values;
2nd motion vector detection means for detecting motion vectors of the shape signal by comparing the shape signal of the input signal and the shape signal of the reference image;
2nd motion compensation means for motion-compensating the shape signal of the reference image using the motion vectors of the shape signal, and outputting the same as a compensated shape signal;
2nd difference value calculation means for calculating from the shape signal of the input signal and the compensated shape signal the difference therebetween, and outputting the same as 2nd difference values;
2nd encoding means for encoding the 2nd difference values; and
3rd encoding means for encoding the motion vectors of the pixel value signal and the motion vectors of the shape signal,
whereby the encoding efficiency is improved as well as the motion compensation errors are further reduced by using appropriate signals which are encoded and decoded as reference images and to which the motion compensation values are added.
A 11th aspect of this invention provides an image encoding apparatus according to aspect 10 wherein the 2nd motion vector detection means detects the motion vectors of the shape signal by comparing the shape signal of the input signal and the shape signal of the reference image in the vicinity of the motion vectors detected by the 1st motion vector detection means,
whereby the calculation times of the motion detection are reduced.
A 12th aspect of this invention provides an image encoding apparatus according to aspect 10 wherein the 1st motion vector detection means detects the motion vectors of the pixel value signal by comparing the pixel value signal of the input signal and the pixel value signal of the reference image in the vicinity of the motion vectors detected by the 2nd motion vector detection means,
whereby because of the result of the motion detection in the transparency signal is used in the motion detection of the pixel value signal, the calculation times of the motion detecting are reduced.
A 13th aspect of this invention provides an image encoding apparatus according to aspect 10 wherein the 3rd encoding means encodes the motion vectors of the pixel value signal and the difference values between the motion vectors of the shape signal and the motion vectors of the pixel value signal,
whereby because the difference vectors between the motion vectors of the pixel value signal and the motion vectors of the shape signal are encoded instead of the motion vectors of the shape signal being encoded, the variable-length encoding enables further improvement in the encoding efficiency.
A 14th aspect of this invention provides an image encoding apparatus according to aspect 10 wherein the 3rd encoding means encodes the motion vectors of the shape signal and the difference values between the motion vectors of the shape signal and the motion vectors of the pixel value signal,
whereby because the difference vectors between the motion vectors of the shape signal and the motion vectors of the pixel value signal are encoded instead of the motion vectors of the pixel value signal being encoded, the variable-length encoding enables further improvement in the encoding efficiency.
A 15th aspect of this invention provides an image encoding apparatus according to aspect 10 wherein,
when the shape signal of the input signal indicates that all pixel values are significant, and the compensated shape signal which is motion-compensated for the shape signal of the reference image using the motion vectors of the pixel value signal detected by the 1st motion vector detection means, indicates that all pixel values are significant, or
when the shape signal of the input signal indicates that all pixel values are insignificant, and the compensated shape signal which is motion-compensated for the shape signal of the reference image using the motion vectors of the pixel value signal detected by the 1st motion vector detection means, indicates that all pixel values are insignificant,
the 2nd vector detection means does not detect the motion vectors of the shape signal, and
the motion vectors of the pixel value signal detected by the 1st vector detection means is used as the motion vectors of the shape signal,
whereby the motion detection of the shape signal when the necessity is therefore low is not performed, resulting in reduced processing load.
A 16th aspect of this invention provides an image encoding apparatus according to aspect 10 wherein,
when the compensated shape signal which is motion-compensated for the shape signal of the reference image using the motion vectors of the pixel value signal detected by the 1st motion vector detection means is compared to the shape signal of the input signal and the resulting difference is lower than the given tolerance value,
the 1st vector detecting means does not detect the motion vector of the pixel value signal but the motion vectors of the shape signal detected by the 2nd vector detection means is used as the motion vectors of the pixel value signal,
whereby the motion detection of the shape signal when the necessity is therefore low is not performed, resulting in reduced processing load.
A 17th aspect of this invention provides an image encoding apparatus according to aspect 10 wherein,
when the motion vectors of the shape signal have been encoded from the immediately previous encoded input signal, the 3rd encoding means encodes the difference values between the immediately previous encoded motion vectors of the shape signal and the motion vectors of the shape signal detected from the input signal when the motion vector of the shape signal of the input signal is encoded in the immediately previous encoded input signal. That is, when the immediately previous motion vectors of the shape signal have been encoded, the difference vectors between the motion vectors and the detected motion vectors are obtained and encoded,
whereby it is possible to improve the encoding efficiency using the motion vectors between shape signals having high correlation.
An 18th aspect of this invention provides an image encoding apparatus according to aspect 10 wherein,
when the immediately previous motion vectors of the pixel value signal of the input signal have been encoded, the 3rd encoding means encodes the difference values between the motion vectors of the immediately previous encoded pixel value signal and the motion vectors of the pixel value signal detected from the input signal. That is, when the immediately previous motion vectors of the pixel value signal have been encoded, the difference vectors between the motion vectors and the detected motion vectors are obtained and encoded,
whereby it is possible to improve the encoding efficiency using differences of the motion vectors between pixel value signals having high correlation.
A 19th aspect of this invention provides an image encoding apparatus according to any of aspects 10 to 18 wherein,
when the input signal consists of the transparency information indicating the synthesis ratio for synthesizing a plurality of images, and the image information, as the transparency information regarded is the shape signal and as the image information regarded is the pixel value signal,
whereby it is possible to improve the encoding efficiency for image signal including transparency information.
A 20th aspect of this invention provides an image encoding apparatus according to any of aspects 10 to 18 wherein,
when the input signal consists of the transparency information indicating the synthesis ratio for synthesizing a plurality of images and the image information, the transparency information is separated into the two-valued signal representing only the shape and the other remaining signal, and then the two-valued signal is regarded as the above shape signal, and the separated remaining shape signal and the image information are regarded as the pixel value signal,
whereby it is possible to improve the encoding efficiency for image signals including transparency information.
A 21st aspect of this invention provides an image encoding apparatus which receives the image signal which consists of at least either the shape information indicating whether pixel values of respective pixels of an object are significant or not, or the transparency information indicating the synthesis ratio for respective pixels of the object, and of the image information, as input image signal, comprising:
blocking means for integrating pixels which spacially and temporally coincide with the input image signal into a group and outputting the group as blocked information;
1st encoding means for selecting an encoding mode from the given set of encoding modes for each piece of shape information formed into blocks by the blocking means, the transparency information and the pixel value information, and encoding each piece of information in each selected encoding mode;
2nd encoding means for collectively encoding all mode-identifying information each of which indicates the selected mode for each piece of shape information, the transparency information and the pixel value information; and
the output of the 1st encoding means and the output of the 2nd encoding means being output as coded outputs;
whereby because all the high-correlated shape information, transparency information and pixel value information are collectively encoded, the variable-length encoding which produces codes having short code lengths for the codes being the same modes makes it possible to reduce the bit number of the encoded mode signal.
A 22nd aspect of this invention provides an image encoding apparatus which receives the image signal which consists of at least either the shape information indicating whether or not pixel values of respective pixels of an object are significant, or the transparency information indicating the synthesis ratio for respective pixels of the object, and the pixel value information, as input image signal, comprising:
blocking means for integrating pixels which spacially and temporally coincide with the input image signal into a group and outputting the group as blocked information;
1st encoding means for selecting an encoding mode from the given set of encoding modes for each piece of shape information formed into blocks by the blocking means and the transparency information, and encoding each piece of information in each selected encoding mode;
2nd encoding means for encoding the pixel value information which is blocked by the blocking means in either of the encoding modes selected by the 1st encoding means; and
3rd encoding means for collectively encoding all mode-identifying information each of which indicates the selected mode for each piece of shape information, the transparency information and the pixel value information,
whereby the selected modes are likely to become identical each other, the outputs of the 1st, 2nd and 3rd encoding means being output as the encoded output, and the variable-length encoding makes it possible to reduce the bit number of the encoded mode signal to a further extent.
A 23rd aspect of this invention provides an image encoding apparatus which receives the image signal which consists of at least either the shape information indicating whether or not pixel values of respective pixels of an object are significant, or the transparency information indicating the synthesis ratio for respective pixels of the object as the input image information, and the pixel value information, as input image signal, comprising:
blocking means for integrating pixels which spacially and temporally coincide with the input image signal into a group and outputting the group as blocked information;
1st encoding means for selecting an encoding mode from the given set of encoding modes for each piece of pixel value information formed into blocks by the blocking means, and encoding each piece of information in each selected encoding mode;
2nd encoding means for encoding the shape information and the transparency information formed into blocks by the blocking means; and
3rd encoding means for collectively encoding all mode-identifying information each of which indicates the selected mode for the shape information, the transparency information and the pixel value information,
whereby because the selected modes are is likely to become identical each other, the outputs of the 1st, 2nd and 3rd encoding means being output as the encoded output, the variable-length encoding makes it possible to further reduce the bit number of the encoded mode signal.
A 24th aspect of this invention provides an image encoding apparatus according to any of aspects 21 to 23 wherein,
the given encoding modes are the intra-frame encoding and the inter-frame encoding,
whereby the encoding based on the correlation of the image signal is performed, and thereby reduction in the bit number of the encoded mode signal is enabled.
A 25th aspect of this invention provides an image encoding apparatus according to any of aspects 21 to 23 wherein,
the 2nd encoding means selects the intra-frame encoding when the selected encoding mode is the intra-frame encoding mode in the 1st encoding means,
whereby the encoding based on the correlation of the image signal is performed using the same modes, and the reduction in the bit number of the encoded mode signal is enabled.
A 26th aspect of this invention provides an image encoding apparatus according to any of aspects 21 to 23 wherein,
the given encoding modes are the number of motion vectors of each of the blocks,
whereby the encoding corresponding to the nature of the image signal is performed, and the reduction in the bit number of the encoded mode signal is enabled.
A 27th aspect of this invention provides an image encoding apparatus according to any of aspects 22 to 23 wherein,
the 2nd encoding means selects the number of motion vectors of each of the blocks which is the encoding mode selected in the 1st encoding means as the encoding mode,
whereby the encoding corresponding to the nature of the image signal is performed using the same modes, and a further reduction in the bit number of the encoded mode signal is enabled.
A 28th aspect of this invention provides an image encoding apparatus according to any of aspects 21 to 23 wherein,
the given encoding modes are the changing of the quantizing step and the non-changing of the quantizing step,
whereby the encoding corresponding to the nature of the image signal is performed, and the reduction in the bit number of the encoded mode signal is enabled.
A 29th aspect of this invention provides an image encoding apparatus according to any of aspects 22 to 23 wherein,
the 2nd encoding means selects the non-changing of the quantizing step when the 1st encoding means selects the non-changing of the quantizing step,
whereby the encoding corresponding to the nature of the image signal is performed, and the reduction in the bit number of the encoded mode signal is enabled.
A 30th aspect of this invention provides an image encoding apparatus which receives two-dimensional image signals consisting of a plurality of pixels as input signals and encodes the image signals, comprising:
1st change pixel detection means for detecting the pixels changing the pixel values by scanning in the given direction on the two-dimensional image signal and outputting the result as the detected 1st change pixels;
2nd change pixel detection means for detecting the pixels changing the pixel values by scanning in the given direction on the already encoded and decoded pixels and outputting the result as the detected 2nd change pixels;
3rd change pixel detection means for detecting the pixels changing the pixel values by scanning in the given direction on the already encoded and decoded pixels and outputting the result as the detected 3rd change pixels;
change pixel prediction means for predicting the 1st change pixels based on the 1st and 2nd change pixels and outputting the result as predicted change pixels;
prediction error calculation means for calculating the differences between the detected 1st change pixels and predicted change pixels, and outputting difference values of change pixels; and
prediction error encoding means for encoding the difference values of change pixels,
whereby the error concerning the prediction is encoded and the improvement in the encoding efficiency is enabled.
A 31st aspect of this invention provides an image encoding apparatus according to aspect 30 wherein,
the 2nd change pixel detection means and the 3rd change pixel detection means makes the pixel values of the 2nd change pixel and the 3rd change pixel equal to that of the 1st change pixel,
whereby the above-described encoding is performed and the above-described effect is obtained.
A 32nd aspect of this invention provides an image encoding apparatus according to aspect 30 wherein,
the 2nd change pixel detection means and the 3rd change pixel detection means use the same given scanning direction as the given scanning direction of the 1st change pixel detection means,
whereby the above-described encoding is performed and the above-described effect is obtained.
A 33rd aspect of this invention provides an image encoding apparatus according to aspect 30 wherein,
the 3rd change pixel is encoded by the difference with the change pixel which is predicted using the 2nd change pixel,
whereby the above-described encoding is performed and the above-described effect is obtained.
A 34th aspect of this invention provides an image encoding apparatus according to aspect 30 wherein,
it is assumed that the 2nd change pixel, 3rd change pixel and 1st change pixel should be on different scanning lines,
whereby the above-described encoding is performed and the above-described effect is obtained.
A 35th aspect of this invention provides an image encoding apparatus according to aspect 30 wherein,
when the 2nd change pixel is the x-th pixel on the m-th scanning line and the 3rd change pixel is the y-th pixel on the n-th scanning line, the change pixel prediction means predicts that the 1st change pixel should be the yxe2x88x92(xxe2x88x92y)*(nxe2x88x92k)/(mxe2x88x92n)-th pixel on the k-th scanning line,
whereby the above-described encoding is performed and the above-described effect is obtained.
A 36th aspect of this invention provides an image encoding apparatus which receives two-dimensional image signals consisting of a plurality of pixels as input signals and encodes the image signals, comprising:
change pixel detection means for detecting the pixels changing the pixel values by scanning the two-dimensional image signal in the given direction to output the result as the detected change pixels;
change pixel prediction means for predicting change pixels based on encoded and decoded pixels and outputting the result as predicted change pixels;
prediction error calculation means for calculating the differences between the detected change pixels and the predicted change pixels, and outputting difference values of change pixels;
prediction error encoding means for encoding the difference value of change pixels and outputting it as a difference value encoded signal, when the difference value of change pixels is less than the given value; and
pixel number encoding means for calculating the number of the pixels which are positioned between the immediately previous encoded change pixel and the detected change pixel and are not positioned at the pixel position which the prediction error encoding means can encode, and encoding the calculated pixel number, and outputting the pixel number encoded signal, when the difference value of changed pixels is equal to or larger than the given value,
the prediction error encoding means and the pixel number encoding means performing encodings in which the encoded signal and the pixel number encoded signal are uniquely identifiable thereby to output the prediction error encoded signal when the prediction error is within the given range and to output the pixel number encoded signal when the prediction error is beyond the given range as the output encoded signal,
whereby when the prediction error is large, the appropriate encoding is carried out, averting the reduction in the encoding efficiency even when the prediction error is so large that the variation in the number of the change pixels unables the prediction of the change pixel.
A 37th aspect of this invention provides an image encoding apparatus according to aspect 36 wherein,
in the prediction error encoding means and the pixel number encoding means, the given value which is to be compared with the difference value of the change pixels is set using the pixel number of the scanning line,
whereby the above-described encoding is performed and the above-described effect is obtained.
A 38th aspect of this invention provides an image encoding apparatus which receives two-dimensional shape signals indicating the area where the pixels representing an object exist as input and encodes the shape signals, comprising:
significant area extracting means for extracting the significant area which contains the pixels representing an object from the shape signal to output a significant area range representing the range covering the extracted significant area;
blocking means for dividing the shape signal into blocks having a plurality of pixels;
shape encoding means for judging whether the blocks output by the blocking means contain the significant area each by each, and encoding at least the significant area of the block to output a shape encoded signal when it is judged that the block contains the significant area; and
the significant area range and the shape encoded signal being made as encoded signal, thereby the range of the significant area is detected and the block size of the shape signal is changed so that only the inside of the significant area of the shape signal is encoded;
whereby the encoding is not performed beyond the range of the significant area, resulting in improved encoding efficiency of the shape signal.
A 39th aspect of this invention provides an image encoding apparatus according to aspect 38 wherein,
the shape encoding means extracts the minimum rectangular area containing the significant area from the blocks produced by the blocking means, and encodes only the inside of the extracted rectangular area,
whereby the above-described encoding is performed and the encoding efficiency is obtained.
A 40th aspect of this invention provides an image encoding apparatus which receives two-dimensional image signals consisting of a plurality of pixels as input signals and encodes the two-dimensional image signals, comprising:
image signal separation means for separating the image signal into at least 2 image signals, and outputting the separated image signals as two or more partial image signals;
1st image signal encoding means for selecting at least one of the partial image signals as target partial image signals and encoding the selected target partial image signal and outputting the result as 1st encoded signal;
prediction probability calculating means for predicting the non-target partial image signals which are the partial image signal other than the target partial image signal, and calculating the probability that the prediction comes true, and outputting the calculated prediction probability; and
2nd image signal encoding means for determining the degree of priority of decoding based on the prediction probability calculated by the prediction probability calculation means, and encoding the non-target partial image signal using the encoding method based on the determined degree of priority,
whereby the smaller prediction probability pixels are encoded with priority, thereby making it possible to realize a hierarchical encoding having little picture-quality degradation without any additional information.
A 41st aspect of this invention provides an image encoding apparatus according to aspect 40 wherein,
the 2nd image signal encoding means determines the degree of priority of decoding so that the small prediction probability pixels are encoded with priority,
whereby the above-described encoding is performed and the encoding efficiency is obtained.
A 42nd aspect of this invention provides an image encoding apparatus according to aspect 40 wherein,
the prediction probability calculating means makes the coming-true probability be large when the pixel values of the neighbor pixels have the same value, while makes the coming-true probability be small when the pixel values of the neighbor pixels do not have the same value,
whereby the above-described encoding is performed and the encoding efficiency is obtained.
A 43rd aspect of this invention provides an image decoding apparatus which receives the encoded signals and decodes the same, comprising:
decoding means for decoding the encoded signal to obtain the encoded mode and the difference value, and outputting the obtained encoding mode as a mode signal and the obtained difference value as a decoded difference value;
1st prediction means for predicting the change pixel of the input signal based on the pixel changing the pixel value among the already encoded and decoded pixels in the frame, and outputting the predicted pixels as 1st predicted pixels;
2nd prediction means for predicting the change pixel of the input signal, with the motion compensation, based on the pixel changing the pixel value among the already encoded and decoded pixels in the reference frame, and outputting the predicted pixels as 2nd predicted pixels;
addition means for adding the decoded difference value to the 1st predicted pixel when the mode signal indicates the prediction of the particular frame, and adding the decoded difference value to the 2nd predicted pixel when the mode signal indicates the prediction of reference frame; and
the output of the addition means being made the change pixel;
whereby the encoded signal obtained by the image encoding apparatus of aspect 2 can be appropriately decoded.
A 44th aspect of this invention provides an image decoding apparatus which receives encoded signals and decodes the same, comprising:
decoding means for decoding the encoded signal to obtain the encoded mode and the difference value, and outputting the obtained encoding mode as a mode signal and the obtained difference value as a decoded difference value;
1st prediction means for predicting the change pixel of the input signal, based on the pixel changing the pixel value among the already encoded and decoded pixels, by horizontally scanning the image signal, and outputting the predicted pixels as 1st predicted pixels;
2nd prediction means for predicting the change pixel of the input signal, with the motion compensation, based on the pixel changing the pixel value among the already encoded and decoded pixels, by scanning the image signal in the vertical direction, and outputting the predicted pixels as 2nd predicted pixels;
addition means for adding the decoded difference value to the 1st predicted pixel when the mode signal indicates the prediction by the horizontal scanning, and adding the decoded difference value to the 2nd predicted pixel when the mode signal indicates the prediction by the vertical scanning; and
the output of the addition means being made the change pixel;
whereby the encoded signal obtained by the image encoding apparatus of aspect 3 can be appropriately decoded.
A 45th aspect of this invention provides an image decoding apparatus which receives encoded signals and decodes the same, comprising:
decoding means for decoding the encoded signal, obtaining the difference value and the pixel value of the change pixel, and outputting the obtained difference value as a decoded difference value and the obtained change pixel as decoded pixel;
prediction means for predicting the change pixel of the input encoded signal, based on the pixel changing the pixel value among the already decoded pixels, and outputting the predicted pixels as predicted pixels;
addition means for adding the decoded difference value to the predicted pixel and outputting the calculated result as a modified difference value; and
image decoding means for obtaining multi-valued signals by a decoding process from the modified difference values and the decoded pixel values,
whereby the encoded signal obtained by the image encoding apparatus of aspect 4 can be appropriately decoded.
A 46th aspect of this invention provides an image decoding apparatus which receives encoded signals and decodes the same, comprising:
1st decoding means for decoding the encoded signal to obtain the difference value of the pixel value signal, and outputting the obtained difference value as a decoded pixel value difference value;
2nd decoding means for decoding the encoded signal to obtain the difference value of the transparency signal, and outputting the obtained difference value as a decoded transparency difference value;
3rd decoding means for decoding the encoded signal to obtain the motion vectors of the pixel value signal and the motion vectors of the transparency signal, and outputting the decoded pixel value motion vectors and the decoded pixel value motion vectors;
1st motion compensation means for compensating the pixel value signal of a reference image described below using the encoded pixel value motion vectors, and outputting the result of the motion compensation as a compensated pixel value signal;
1st addition means for adding the decoded pixel value difference value and the compensated pixel value signal to output the result of the addition as a decoded pixel value signal as well as a pixel value of a reference image;
2nd motion compensation means for compensating the transparency signal of a reference image described-below using the decoded transparency motion vectors, and outputting the result of the motion compensation as a compensated transparency signal; and
2nd addition means for adding the decoded transparency difference value and compensated transparency signal, and outputting the result of the addition as a decoded transparency signal as well as a transparency signal of a reference image;
whereby the encoded signal obtained by the image encoding apparatus of aspect 5 can be appropriately decoded.
A 47th aspect of this invention provides an image decoding apparatus according to aspect 46 wherein,
the 3rd decoding means decodes the motion vector of the pixel value signal and the difference value of the motion vectors to obtain the decoded pixel value motion vectors and the decoded motion vector difference values, and adding the decoded pixel value motion vectors and the decoded motion vector difference values, and making the result of the addition be the encoded transparency motion vectors,
whereby the encoded signal obtained by the image encoding apparatus of aspect 7 can be appropriately decoded.
A 48th aspect of this invention provides an image decoding apparatus which receives encoded signals and decodes the same, comprising:
1st decoding means for decoding the encoded signal to obtain the difference value of the pixel value signal, and outputting the obtained difference value as a decoded pixel value difference value;
2nd decoding means for decoding the encoded signal to obtain the difference value of the shape signal, and outputting the obtained difference value as a decoded shape difference value;
3rd decoding means for decoding the encoded signal to obtain the motion vectors of the pixel value signal and the motion vectors of the shape signal, and outputting the decoded pixel value motion vectors and the decoded shape motion vectors;
1st motion compensation means for compensating the pixel value signal of a reference image described below using the encoded pixel value motion vectors, and outputting the result of the motion compensation as a compensated pixel value signal;
1st addition means for adding the decoded pixel value difference value and the compensated pixel value signal, and outputting the result of the addition as a decoded pixel value signal as well as a pixel value signal of a reference image;
2nd motion compensation means for compensating the shape signal of a reference image described below using the decoded shape motion vectors, and outputting the result of the motion compensation as a compensated shape signal; and
2nd addition means for adding the decoded shape difference value and the compensated shape signal, and outputting the result of the addition as a decoded shape signal as well as a shape signal of a reference image,
whereby the encoded signal obtained by the image encoding apparatus of aspect 10 can be appropriately decoded.
An image decoding apparatus according to a 49th aspect of this invention is an image decoding apparatus according to aspect 48 wherein,
the 3rd decoding means decodes the motion vector of the pixel value signal and the difference value of the motion vectors to obtain the decoded pixel value motion vectors and the decoded motion vector difference values, and adding the decoded pixel value motion vectors and the decoded motion vector difference values, and making the result of the addition be the encoded shape motion vectors,
whereby the encoded signal obtained by the image encoding apparatus of aspect 13 can be appropriately decoded.
A 50th aspect of this invention provides an image decoding apparatus according to aspect 48 wherein,
the 3rd decoding means decodes the motion vector of the shape signal and the difference value of the motion vectors to obtain the decoded shape motion vectors and the decoded motion vector difference values, and adding the decoded shape motion vectors and the decoded motion vector difference values, and making the result of the addition be the encoded pixel value motion vectors,
whereby the encoded signal obtained by the image encoding apparatus of aspect 14 can be appropriately obtained.
A 51st aspect of this invention provides an image decoding apparatus according to aspect 48 wherein,
when the input signal is one which is obtained by encoding the difference value between the immediately previous encoded motion vectors of the shape signal and the motion vectors of the shape signal detected from the input signal, the 3rd decoding means decodes the difference value, and adds the decoded difference value to the immediately previous decoded motion vector of the shape signal to obtain the shape motion vector,
whereby the encoded signal obtained by the image encoding apparatus of aspect 17 can be appropriately obtained.
A 52nd aspect of this invention provides an image decoding apparatus according to aspect 48 wherein,
when the input signal is one which is obtained by encoding the difference value between the immediately previous encoded motion vector of the pixel value signal and the motion vector of the pixel value signal detected from the input signal, the 3rd decoding means decodes the difference value, and adds the decoded difference value to the immediately previous decoded motion vector of the pixel value signal to output the decoded pixel value motion vector,
whereby the encoded signal obtained by the image encoding apparatus of aspect 18 can be appropriately decoded.
A 53rd aspect of this invention provides an image decoding apparatus according to any of aspects 48 to 52 wherein,
when the input signal is one consisting of the transparency information indicating the synthesis ratio for synthesizing a plurality of images, and the image information, the decoded shape signal is made the transparency information and the decoded pixel value signal is made the image information,
whereby the encoded signal obtained by the image encoding apparatus of aspect 19 can be appropriately decoded.
A 54th aspect of this invention provides an image decoding apparatus according to any of aspects 48 to 52 wherein,
when the input signal is an encoded signal created from an image signal which consists of the transparency information indicating the synthesis ratio for synthesizing a plurality of images, and the image information, the transparency information of which is separated into the two-valued signal representing only the shape and the other remaining shape signal, then, the two-valued signal is made the shape signal, and the separated remaining shape signal and image information are made the pixel value signal to be encoded, the decoded shape signal is made the two-valued signal of the transparency information, and the decoded pixel value signal and the image information are made the remaining shape signal of the transparency information,
whereby the encoded signal obtained by the image encoding apparatus of aspect 20 can be appropriately decoded.
A 55th aspect of this invention provides an image decoding apparatus which receives encoded signals and decodes the same, comprising:
1st decoding means for decoding the input encoded signal to obtain the mode identifying information indicating each encoding mode of the shape information, the transparency information and the pixel value information;
2nd decoding means for decoding, according to the obtained mode identifying information, the blocked shape information, transparency information and pixel value information; and
reverse blocking means for integrating the blocked shape information, transparency information and pixel value information output by the 2nd decoding means to output the decoded image signals,
whereby the encoded signal obtained by the image encoding apparatus of aspect 21 can be appropriately decoded.
A 56th aspect of this invention provides an image decoding apparatus according to aspect 55 wherein,
the mode identifying information indicates the intra-frame encoding and the inter-frame encoding as encoding modes,
whereby the encoded signal obtained by the image encoding apparatus of aspect 24 can be appropriately decoded.
A 57th aspect of this invention provides an image decoding apparatus according to aspect 55 wherein,
the mode identifying information indicates the number of motion vectors of each of the blocks as encoding modes,
whereby the encoded signal obtained by the image encoding apparatus of aspect 26 can be appropriately decoded.
A 58th aspect of this invention provides an image decoding apparatus wherein,
the mode identifying information indicates the changing and non-changing of the quantizing step as encoding modes,
whereby the encoded signal obtained by the image encoding apparatus of aspect 28 can be appropriately decoded.
A 59th aspect of this invention provides an image decoding apparatus which receives the two-dimensional image signals consisting of a plurality of pixels as input encoded signals and decoded the same to output, comprising:
2nd change pixel detection means for detecting the pixels changing the pixel values by scanning the already decoded pixels in the given direction to output the result as the detected 2nd change pixels;
3rd change pixel detection means for detecting the pixels changing the pixel values by scanning the already decoded pixels in the given direction to output the result as the detected 3rd change pixels;
change pixel prediction means for predicting 1st change pixels described below based on the 2nd and 3rd change pixels to output the result as predicted change pixels;
prediction error decoding means for decoding the input encoded signals to obtain the prediction error and outputting the obtained prediction error;
1st change pixel decoding means for adding the predicted change pixel and the prediction error to output the result of the addition as a 1st change pixel; and
pixel value decoding means for decoding the pixel values of the particular pixels, assuming that there should be no pixels changing pixel values between the immediately previous decoded change pixel and the 1st change pixel,
whereby the encoded signal obtained by the image encoding apparatus of aspect 30 can be appropriately decoded.
A 60th aspect of this invention provides an image decoding apparatus according to aspect 59 wherein,
the 2nd change pixel detection means and the 3rd change pixel detection means use those the same as that of the 1st change pixel, as the pixel values of the 2nd change pixel and the 3rd change pixel respectively,
whereby the encoded signal obtained by the image encoding apparatus of aspect 31 can be appropriately decoded.
A 61st aspect of this invention provides an image decoding apparatus according to aspect 59 wherein,
the 2nd change pixel detection means and the 3rd change pixel detection means use the same given scanning direction as the given scanning direction of the 1st change pixels detection means,
whereby the encoded signal obtained by the image encoding apparatus of aspect 32 can be appropriately decoded.
A 62nd aspect of this invention provides an image decoding apparatus according to aspect 59 wherein,
3rd change pixel is decoded by the difference with the change pixel which is predicted using the 2nd change pixel,
whereby the encoded signal obtained by the image encoding apparatus of aspect 33 can be decoded.
A 63rd aspect of this invention provides an image decoding apparatus according to aspect 34 wherein,
it is assumed that the 2nd change pixel, the 3rd change pixel and the 1st change pixel should be on different scanning lines,
whereby the encoded signal obtained by the image encoding apparatus of aspect 34 can be appropriately decoded.
A 64th aspect of this invention provides an image decoding apparatus according to aspect 59 wherein,
when the 2nd change pixel is the x-th pixel on the m-th scanning line and the 3rd change pixel is the y-th pixel on the n-th scanning line, the change pixel prediction means predicts that the 1st change pixel should be the yxe2x88x92(xxe2x88x92y)*(nxe2x88x92k)/(mxe2x88x92n)-th pixel on the k-th scanning line,
whereby the encoded signal obtained by the image encoding apparatus of aspect 35 can be appropriately decoded.
A 65th aspect of this invention provides an image decoding apparatus which receives the two-dimensional image signals consisting of a plurality of pixels as encoded input signals, and decodes the same to output, comprising:
change pixel detection means for detecting the pixels changing the pixel values by scanning the already decoded two-dimensional image signal in the given direction to output the result as the detected change pixels;
change pixel prediction means for predicting change pixels on the particular scanning line based on the detected change pixels and outputting the result as predicted change pixels;
mode decoding means for decoding the input encoded signals and judging whether the input signal is the difference value encoded signal or pixel number encoded signal, and outputting the identifying signal;
prediction error decoding means for decoding the difference value encoded signal and outputting the decoded prediction error when the identifying signal indicates the difference value encoded signal;
1st change pixel decoding means for adding the predicted change pixel and the encoded prediction error, and outputting the result of the addition as a 1st change pixel, when the identifying signal indicates the difference value encoded signal;
2nd change pixel decoding means for decoding the number of pixels which should be positioned from the immediately previous encoded change pixel to the detected change pixel, from the pixel number encoded signal to obtain the position of the change pixel on the basis of the number of the decoded pixels, and outputting the obtained result as the 2nd decoded change pixel, when the identifying signal indicates the difference value signal;
change pixel selection means for selecting the 1st decoded change signal or the 2nd decoding change pixel according to the identifying signal; and
change pixel decoding means for decoding the pixel values, assuming that there should be no pixels changing pixel values between the immediately previous decoded change pixel and the 1st change pixel;
whereby the encoded signal obtained by the image encoding apparatus of aspect 36 is appropriately decoded.
A 66th aspect of this invention provides an image decoding apparatus wherein,
the change pixel selection means performs the selection according to the pixel number on the particular scanning line,
whereby the encoded signal obtained by the image encoding apparatus of aspect 37 is appropriately obtained.
A 67th aspect of this invention provides an image decoding apparatus which receives encoded signals and decodes the same to output two-dimensional shape signals representing the area where there exist the pixels representing an object, comprising:
significant area decoding means for decoding the encoded signals, obtaining rectangular areas where there exist the pixels representing an object and outputting the obtained areas as significant areas;
shape decoding means for judging whether or not each of blocks consisting of a plurality of pixels contains the significant area, and when judging that the block contains the significant area, encoding at least the significant area of the particular block to output the decoded result as decoded block shape signals; and
reverse blocking means for integrating the decoded block shape signals to constitute a two-dimensional shape signal, and outputting the two-dimensional shape signal as a decoded signal,
whereby the encoded signal obtained by the image encoding apparatus of aspect 38 can be decoded.
A 68th aspect of this invention provides an image decoding apparatus according to aspect 67 wherein,
the shape decoding means extracts the minimum rectangular area containing the significant area from the block for each block and decoding only the inside of the extracted rectangular area,
whereby the encoded signal obtained by the image encoding apparatus of aspect 39 can be appropriately obtained.
A 69th aspect of this invention provides an image decoding apparatus which receives encoded signals, and decodes the same to output two-dimensional image signals consisting of a plurality of pixels, comprising:
1st image signal decoding means for decoding the encoded signals to output 1st decoded signals;
image prediction means for predicting and outputting image signals which have not been decoded by the 1st image signal decoding means, based on the image signals decoded by the 1st image signal decoding means;
prediction probability calculation means for calculating the probability that the predicted image signal comes true to output the same;
2nd image signal decoding means for decoding the encoded signals which are input with the degree of priority according to the prediction probability calculated by the prediction probability calculation means; and
decoded signal integration means for integrating the outputs of the 1st image signal decoding means and the 2nd image signal decoding means, and replacing the image signal which is not decoded by any of the 1st image signal decoding means and the 2nd image signal decoding means, with the image signal predicted by the image prediction means, and outputting the result as a decoded image signal,
whereby the encoded signal obtained by the image encoding apparatus of aspect 40 can be appropriately decoded.
A 70th aspect of this invention provides an image decoding apparatus according to aspect 69 wherein,
the 2nd image signal decoding means gives priority of the decoding of the pixel with the small prediction probability,
whereby the encoded signal obtained by the image encoding apparatus of aspect 41 can be appropriately decoded.
A 71st aspect of this invention provides an image decoding apparatus according to aspect 69 wherein,
the prediction probability calculating means makes the coming-true probability large when the pixel values of the neighbor pixels have the same value, while makes the coming-true probability small when the pixel values of the neighbor pixels have different values,
whereby the encoded signal obtained by the image encoding apparatus of aspect 42 can be appropriately decoded.
A 72nd aspect of this invention provides an image encoding method which receives two-valued image signals as input signals and encodes pixels of the input signals changing the pixel values, comprising:
change pixel detection step for detecting the pixels changing the pixel values and outputting the result as the detected change pixels;
1st prediction step for predicting change pixels of the input signals based on the pixels changing the pixel values of already encoded and decoded pixels in the particular frames and outputting the result as 1st predicted pixels;
1st difference value calculation step for calculating the differences between the detected change pixels and the 1st predicted change pixels to output the calculated difference as 1st difference values D;
2nd prediction step for predicting change pixels of the input signals, along with the motion compensation, based on the pixels changing the pixel values of already encoded and decoded pixels in reference frames to output the predicted result as 2nd predicted pixels;
2nd difference value calculation step for calculating the differences between the detected change pixels and the 2nd predicted change pixels to output the calculated difference as 2nd difference values Dxe2x80x3;
mode selection step for calculating the code lengths of the first difference value D and the second difference value Dxe2x80x3 when respectively encoded, and selecting the value having the shorter code length by comparing the calculated results, and outputting either of xe2x80x9cthe firstxe2x80x9d or xe2x80x9cthe secondxe2x80x9d depending on the selection as an encoding mode; and
encoding step for encoding the selected first difference value D or the second difference values Dxe2x80x3, and the encoded mode output by the mode selection mode,
whereby an encoded signal which should have the minimum code length can be selected and output by comparing the prediction based on the particular frame and the prediction based on the motion-compensated reference frame to perform encoding, resulting in reduced bit number required for encoding by utilizing the inter-frame pixel correlation.
A 73rd aspect of this invention provides an image encoding method which receives two-dimensional two-valued image signals as input signals and encodes pixels of the input signals changing the pixel values, comprising:
change pixel detection step for detecting the pixels changing the pixel values and outputting the result as the detected change pixels;
1st prediction step for predicting change pixels of the input signals based on the pixels changing the pixel values of already encoded and decoded pixels by horizontally scanning the image signals and outputting the result as 1st predicted pixels;
1st difference value calculation step for calculating the differences between the detected change pixel and the 1st predicted change pixel and outputting the calculated difference as 1st difference values D;
2nd prediction step for predicting change pixels of the input signals based on the pixels changing the pixel values of already encoded and decoded pixels by vertically scanning the image signals and outputting the predicted pixel as 2nd predicted pixels;
2nd difference value calculation step for calculating the differences between the detected change pixel and 2nd predicted change pixel and outputting the calculated difference as 2nd difference value Dxe2x80x3;
mode selection step for calculating the code lengths of the first difference value D and the second difference value Dxe2x80x3 when respectively encoded, selecting one having the shorter code length by comparing the calculated results, and outputting either of xe2x80x9cthe firstxe2x80x9d or xe2x80x9cthe secondxe2x80x9d as an encoding mode according to the selection; and
encoding step for encoding the selected first difference value D or the second difference value Dxe2x80x3, and the encoded mode output by the mode selection mode,
whereby a signal which should have the minimum code length can be selected and output by comparing the prediction by horizontal scanning and the prediction by vertical scanning to perform encoding, resulting in reduced bit number required for encoding utilizing local changes in the horizontal and vertical correlations of the image.
A 74th aspect of this invention provides an image encoding method which receives image signals with blocked shapes consisting of shape signals indicating the shapes of objects and whether the pixel value of pixels are significant or not and the pixel value signals as input signals and encodes the input signals referring to reference images, comprising:
1st motion vector detection step for detecting motion vectors of the pixel value signal by comparing the pixel value signal of the input signal and the pixel value signal of the reference image;
1st motion compensation step for motion-compensating the pixel value signal of the reference image using the motion vector of the pixel value signal, and outputting a compensated pixel value signal;
1st difference value calculation step for calculating the difference between the pixel value signal of the input signal and the compensated pixel value signal, and outputting 1st difference values;
1st encoding step for encoding the 1st difference value;
2nd motion vector detection step for detecting motion vectors the shape signal by comparing the shape signal of the input signal and shape signal of the reference image;
2nd motion compensation step for motion-compensating the shape signal of the reference image using the motion vector of the shape signal, and outputting a compensated shape signal;
2nd difference value calculation step for calculating the difference between the shape signal of the input signal and the compensated shape signal, and outputting 2nd difference value;
2nd encoding step for encoding the 2nd difference value; and
3rd encoding step for encoding the motion vector of the pixel value signal and the motion vector of the shape signal,
whereby the encoding efficiency is improved and the motion compensation errors are further reduced using more appropriate signals which are obtained from reference images being subjected to encoding and decoding and the motion compensation value being added thereto.
A 75th aspect of this invention provides an image encoding method which receives the image signal which consists of either of the shape information indicating whether or not pixel values of respective pixels of an object are significant, or the transparency information indicating the synthesis ratio for respective pixels of the object, and the pixel value information, as an input image signal, comprising:
blocking step for integrating pixels which spacially and temporally coincide with the input image signal into a group and outputting the group as blocked information;
1st encoding step for selecting an encoding mode from the given set of encoding modes for each piece of shape information formed into blocks by the blocking step, the transparency information and the pixel value information, and encoding each piece of information with each selected encoding mode;
2nd encoding step for encoding all mode-identifying information, each of which indicates the selected mode for each piece of shape information, transparency information, and pixel value information; and
the outputs of the 1st and 2nd encoding step being output as encoded outputs,
whereby all the high-correlated shape, transparency and pixel value information are collectively encoded, and the variable-length encoding in which codes having the same modes have short code length can be used, resulting in reduced bit number of the encoded mode signal.
A 76th aspect of this invention provides an image encoding method which receives the image signal which consists of either of the shape information indicating whether pixel values of respective pixels of an object are significant or not, or the transparency information indicating the synthesis ratio for respective pixel of the object, and the pixel value information, as an input image signal, comprising:
blocking step for integrating pixels which spacially and temporally coincide with the input image signal into a group and outputting the group as blocked information;
1st encoding step for selecting an encoding mode from the given set of encoding modes for each piece of shape information formed into blocks by the blocking step and the transparency information, and encoding each piece of information with each selected encoding mode;
2nd encoding step for encoding the pixel value information formed into blocks by the blocking step with either of the encoding modes selected by the 1st encoding step;
3rd encoding step for encoding all mode-identifying information, each of which indicates the selected mode for each piece of shape information, the transparency information and the pixel value information; and
the outputs of the 1st encoding step, 2nd encoding step and 3rd encoding step being output as the encoded outputs,
whereby the selected modes are is likely to become identical each other, and the variable-length encoding makes it possible to reduce the bit number of the encoded mode signal to a further extent.
A 77th aspect of this invention provides an image encoding method which receives the image signal which consists of either of the shape information indicating whether pixel values of respective pixels of an object are significant or not, or the transparency information indicating the synthesis ratio for respective pixels of the object, and the pixel value information, as an input image signal, comprising:
blocking step for integrating pixels which spacially and temporally coincide with the input image signal into a group and outputting the group as blocked information;
1st encoding step for selecting an encoding mode from the given set of encoding modes for the pixel value information formed into blocks by the blocking step, and encoding the pixel value information with the selected encoding mode;
2nd encoding step for encoding the shape information and the transparency information formed into blocks by the blocking step, and the transparency information with the encoding mode selected in the 1st encoding step;
3rd encoding step for encoding all mode-identifying information, each of which indicates the selected mode for each piece of shape information, the transparency information and the pixel value information; and
the outputs of the 1st, 2nd and 3rd encoding steps being output as the encoded output,
whereby the selected modes become is likely to become identical each other by outputting, and the variable-length encoding makes it possible to reduce the bit number of the encoded mode signal to a further extent.
A 78th aspect of this invention provides an image encoding method which receives two-dimensional image signals consisting of a plurality of pixels as input signals and encodes the same, comprising:
1st change pixel detection step for detecting the pixels changing the pixel values by scanning the two-dimensional image signal in the given direction and outputting the detected 1st change pixels;
2nd change pixel detection step for detecting the pixels changing the pixel values by scanning the already encoded and decoded pixels in the given direction and outputting the detected 2nd change pixels;
3rd change pixel detection step for detecting the pixels changing the pixel values by scanning the already encoded and decoded pixels in the given direction and outputting the detected 3rd change pixels;
change pixel prediction step for predicting the 1st change pixels based on the 2nd change pixels and the 3rd change pixels and outputting the predicted change pixels;
prediction error calculation step for calculating the differences between the 1st change pixels and the predicted change pixels, and outputting the calculated difference values of change pixels; and
prediction error encoding step for encoding the difference values of change pixels to be encoded signals,
whereby the error in prediction is encoded, resulting in enhancement in the encoding efficiency.
A 79th aspect of this invention provides an image encoding method which receives two-dimensional image signals consisting of a plurality of pixels as input signals and encodes the same, comprising:
change pixel detection step for detecting the pixels changing the pixel values by scanning the two-dimensional image signal in the given direction and outputting the detected change pixels;
change pixel prediction step for predicting change pixels based on encoded and decoded pixels and outputting the predicted change pixels;
prediction error calculation step for calculating the differences between the detected change pixels and predicted change pixels, and outputting the calculated difference values of change pixels;
prediction error encoding step for encoding the difference value of change pixels and outputting the difference value encoded signal, when the difference value of change pixels is less than the given value;
pixel number encoding step for calculating the number of the pixels which are positioned between the immediately previous encoded pixel and the above-described detected change pixel and are not positioned at the pixel position where the prediction error encoding step can encode, and encoding the calculated pixel number to output an pixel number encoded signal, when the difference value of change pixel is less than the given value; and
the prediction error encoding step and the pixel number encoding step performing encodings in which the difference value encoded signal and the pixel number encoded signal are uniquely identifiable thereby to output the prediction error encoded signal when the prediction error is within the given range, and to output the pixel number encoded signal when the prediction error is beyond the given range as the output encoded signal;
whereby when the prediction error is large, the appropriate encoding is carried out, averting the reduction in the encoding efficiency even when the prediction error is so large that the variations of the number of the change pixels unables the prediction of the change pixel.
An 80th aspect of this invention provides an image encoding method which receives two-dimensional shape signals indicating the area where pixels representing an object exist and encodes the shape signals, comprising:
significant area extracting step for extracting the significant area which contains the pixels representing an object from the input shape signal and outputting a significant area range representing the range covering the extracted significant area;
blocking step for dividing the shape signal into blocks having a plurality of pixels;
shape encoding step for judging whether each of the respective blocks output by the blocking step contains the significant area, and encoding the significant area of the block at least when it is judged that the block contains the significant area; and
the significant area range and the shape encoded signal being made as encoded signals;
whereby the range of the significant area is detected and the block size of the shape signal is changed so that the shape signal only in the inside of the significant area is encoded, resulting in no encoding being performed beyond the range of the significant area, and improved encoding efficiency for the shape signal.
An 81st aspect of this invention provides an image encoding method which receives two-dimensional image signals consisting of a plurality of pixels as input signals and encodes the same, comprising:
image signal separation step for separating the image signal into at least 2 image signals, and outputting the separated image signals as 2 or more partial image signals;
1st image signal encoding step for selecting at least one of the partial image signals as a target partial image signal and encoding the selected target partial image signal to output a 1st encoded signal;
prediction probability calculating step for predicting the non-target partial image signal which is the partial image signal except the target partial image signal on the basis of the image signal decoded from the 1st encoded signal, and calculating the probability that the prediction comes true, and outputting the calculated prediction probability; and
2nd image signal encoding step for determining the degree of priority of decoding based on the prediction probability calculated by the prediction probability calculation step, and encoding the non-target partial image signal using the encoding method according to the determined degree of priority,
whereby the pixels of smaller prediction probability are encoded with priority, resulting in that the hierarchical encoding having less picture-quality degradation can be performed without any additional information.
An 82nd aspect of this invention provides an image decoding method which receives encoded signals and decodes the same, comprising:
decoding step for decoding the encoded signal to obtain the encoded mode and the difference value, and outputting the obtained encoding mode as a mode signal and the obtained difference value as a decoded difference value;
1st prediction step for predicting the change pixel of the input signal based on the pixel changing the value among the already encoded and decoded pixels in the particular frame, and outputting the predicted pixels as 1st predicted pixels;
2nd prediction step for predicting the change pixel of the input signal, with the motion compensation, based on the pixel changing the value among the already encoded and decoded pixels in the reference frame, and outputting the predicted pixels as 1st predicted pixels;
addition step for adding the decoded difference value to the 1st predicted pixel when the mode signal indicates the frame prediction, and adding the decoded difference value to the 2nd predicted pixel when the mode signal indicates the reference frame prediction; and
the output of the addition step being made the change pixel;
whereby the encoded signal obtained by the image encoding apparatus of aspect 72 can be appropriately decoded.
An 83rd aspect of this invention provides an image decoding method which receives encoded signals and decodes the same, comprising:
decoding step for decoding the encoded signal to obtain the encoded mode and the difference value, and outputting the obtained encoding mode as a mode signal and the obtained difference value as a decoded difference value;
1st prediction step for predicting the change pixel of the input signal, based on the pixel changing the pixel value among the already encoded and decoded pixels, by horizontally scanning the image signal, and outputting the predicted pixels as 1st predicted pixels;
2nd prediction step for predicting the change pixel of the input signal, with the motion compensation, based on the pixel changing the pixel value among the already encoded and decoded pixels, by horizontally scanning the image signal, and outputting the predicted pixels as 2nd predicted pixels;
addition step for adding the decoded difference value to the 1st predicted pixel when the mode signal indicates the prediction by the horizontal scanning, and adding the decoded difference value to the 2nd predicted pixel when the mode signal indicates the prediction by the vertical scanning; and
the output of the addition step being made the change pixel;
whereby the encoded signal obtained by the image encoding apparatus of aspect 73 can be appropriately decoded.
An 84th aspect of this invention provides an image decoding method which receives encoded signals and decodes the same, comprising:
1st decoding step for decoding the encoded signal to obtain the difference value of the pixel value signal, and outputting the decoded pixel value difference value;
2nd decoding step for decoding the encoded signal to obtain the difference value of the shape signal, and outputting the decoded shape difference value;
3rd decoding step for decoding the encoded signal to obtain the motion vectors of the pixel value signal and the motion vectors of the shape signal, and outputting the decoded pixel value motion vectors and the decoded shape motion vectors;
1st motion compensation step for compensating the pixel value signal of a reference image described below using the decoded pixel value motion vectors, and outputting the result of the motion compensation as a compensated pixel value signal;
1st addition step for adding the decoded pixel value difference value and the compensated pixel value signal, and outputting the result of addition as a decoded pixel value signal as well as a pixel value signal of a reference image;
2nd motion compensation step for compensating the shape signal of a reference image described below using the decoded shape motion vectors, and outputting the result of the motion compensation as a compensated shape signal; and
2nd addition step for adding the decoded shape difference value to the compensated shape signal, and outputting the result of the addition as a decoded shape signal as well as a shape signal of a reference image;
whereby the encoded signal obtained by the image encoding apparatus of aspect 74 can be appropriately decoded.
An 85th aspect of this invention provides an image decoding method which receives encoded signals and decodes the same, comprising:
1st decoding step for decoding the input encoded signal to obtain the mode identifying information indicating each encoding mode of the shape information, transparency information and pixel value information;
2nd decoding step for decoding, according to the obtained mode identifying information, the blocked shape information, transparency information and pixel value information; and
reverse blocking step for integrating the blocked shape information, transparency information and pixel value information output by the 2nd decoding step and outputting the decoded image signal,
whereby the encoded signal obtained by the image encoding apparatus of aspect 76 can be appropriately decoded.
An 86th aspect of this invention provides an image decoding method which receives encoded signals as input signals and decodes the two-dimensional image signals consisting of a plurality of pixels, comprising:
2nd change pixel detection step for detecting the pixels changing the pixel values by scanning the already decoded pixels in the given direction and outputting the detected 2nd change pixels;
3rd change pixel detection step for detecting the pixels changing the pixel values by scanning the already decoded pixels in the given direction and outputting the detected 3rd change pixels;
change pixel prediction step for predicting 1st change pixels described below based on the 2nd change pixels and the 3rd change pixels and outputting the predicted change pixels;
prediction error decoding step for decoding the input encoded signals to obtain the prediction error and outputting the obtained prediction error;
1st change pixel decoding step for adding the predicted change pixel and prediction error, and outputting the 1st change pixel; and
pixel value decoding step for decoding the pixel values, assuming that there should be no pixels changing pixel values between the immediately previous decoded change pixel and 1st change pixel,
whereby the encoded signal obtained by the image encoding apparatus of aspect 79 can be appropriately decoded.
An 87th aspect of this invention provides an image decoding method which receives the encoded two-dimensional image signals consisting of a plurality of pixels as input signals, and encodes and outputs the image signals, comprising:
change pixel detection step for detecting the pixels changing the pixel values by scanning the already decoded two-dimensional image signal in the given direction and outputting the detected change pixels;
change pixel prediction step for predicting change pixels on the particular scanning line based on the detected change pixels and outputting the result as predicted change pixels;
mode decoding step for decoding the input encoded signals and judging whether the input signal is the difference value encoded signal or pixel number encoded signal, and outputting the identifying signal;
prediction error decoding step for decoding the difference value encoded signal and outputting the decoded prediction error when the identifying signal indicates the encoded difference value signal;
1st change pixel decoding step for adding the predicted change pixel and the decoded prediction error and outputting the result of the addition as a 1st decoded change pixel, when the identifying signal indicates the encoded difference value signal;
2nd change pixel decoding step for decoding the number of pixels which are not positioned at the pixel positions of the decoded prediction error between the immediately previous decoded change pixel and the detected 1st change pixel from the encoded pixel number signal, and obtaining the positions of change pixels based on the number of the decoded pixels, and outputting the obtained result as 2nd decoded change pixels, when the identifying signal indicates the encoded pixel number value signal;
change pixel selection step for selecting the 1st decoded change pixel or the 2nd decoded change pixel according to the identifying signal to output the same; and
change pixel decoding step for decoding the pixel values, assuming that there should be no pixels changing pixel values between the immediately previous decoded change pixel and the 1st change pixel,
whereby the encoded signal obtained by the image encoding apparatus of aspect 80 can be appropriately decoded.
An 88th aspect of this invention provides an image decoding method which receives encoded signals and decodes the same to output two-dimensional shape signals representing the area where there exist pixels representing an object, comprising:
significant area decoding step for decoding the encoded signals, obtaining rectangular areas where there exist the pixels representing an object, and outputting the obtained areas as significant areas;
shape decoding step for judging whether each of blocks having a plurality of pixels contains the significant area or not, and when it is judged that the block contains the significant area, encoding at least the significant area of the block, and outputting the decoded result as decoded block shape signals; and
reverse blocking step for integrating the decoded block shape signals to constitute a two-dimensional shape signal, and outputting the constituted two-dimensional shape signal,
whereby the encoded signal obtained by the image encoding apparatus of aspect 81 can be appropriately decoded.
An 89th aspect of this invention provides an image decoding method which receives encoded signals and decodes two-dimensional image signals consisting of a plurality of pixels to output the same, comprising:
1st image signal decoding step for decoding the encoded signals and outputting 1st decoded signals;
image prediction step for predicting and outputting image signals which have not decoded by the 1st image signal decoding step, based on the image signals decoded by the 1st image signal decoding step;
prediction probability calculation step for calculating and outputting the probability that the predicted image signal comes true;
2nd image signal decoding step for decoding the encoded signals which are input with the degree of priority according to the prediction probability calculated by the prediction probability calculation step; and
decoded signal integration step for integrating the outputs of the 1st image signal decoding step and the 2nd image signal decoding step, and replacing the image signal which is neither decoded by the 1st image signal decoding step nor by the 2nd image signal decoding step, with the image signal predicted by the image prediction step, and outputting the decoded image signal,
whereby the encoded signal obtained by the image encoding apparatus of aspect 82 can be appropriately decoded.
90th to 99th aspects of this invention provides image encoding program media wherein, programs implementing image encoding methods of the 72nd to 81st aspects are recorded,
whereby image encodings of high encoding efficient are performed on computers which are equipped with these media.
100th to 107th aspects of this invention provides image decoding program media wherein, programs implementing image decoding methods of the 82nd to 189th aspects are recorded,
whereby decoding of the encoded signals obtained by the image encoding methods of the 72nd to 81st aspects can be appropriately performed on computers which are equipped with these media.